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ABSTRACT 

This report provides results of a review of the 
methodology used by the Office of Human Development Services (HDS) to 
measure Head Start performance and to control high risk Head Start 
agencies. The review was performed at HDS headquarters and regional 
locations nationwide. The review was based on a saunple of 200 Head 
Start agencies and focused on Head start performace standards, 
program profiles, the Objective Tracking System, best practice and 
compliance reports, and a regional office's internal procedures for 
the identification of grantees needing assistance. Fiscal and program 
ratings were obtained from HDS headquarters and regional office 
staff, and discussions were held with regional and headquarters 
staff. Methods used by HDS to identify higii risk agencies were 
examined. Title 45 of the Code of Federal Regulations, Part 74, was 
reviewed, as were the Department of Health and Human Services' Grants 
Administration Manual, High Risk Grantees, Chapter l-05, and 
Monitoring Discretionary Grants, Chapter 1-13; and HDS Grants 
Administrative staff Manual, Chapter 8. Findings indicated that HDS 
had not established cr implemented procedures necessary to evaluate 
the performance of Head start agencies or assure that appropriate 
action to upgrade or eliminate high risk agencies would be taken. 
Improvements are recommended. An appendix provides HDS' response. 
(RH) 
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OFFICE OF INSPECTOR GENERAL 

The mission of the Office of Inspector General (OIG), as mandated by Public Law 93- 
452, as amended, is to protect the integrity of the Department of Health and Human 
Sendees* (HHS) programs as well as the health and welfare of beneficiaries served by 
those programs. This statutoiy mission is carried out through a nationwide network of 
audits, investigations, and inspections conducted by three OIG operating components: 
the Office of Audit Services, the Office of Investigations, and the Office of Evaluation 
and Inspections. The OIG also informs the Secretary of HHS of program and 
management problems, and recommends courses to correct them. 

OFFICE OF AUDIT SERVICES 

The OIG's Office of Audit Services (OAS) provides all auditing services for HHS, 
either by conducting audits with its own audit resources or by overseeing audit work 
done by otters. Audits examine the performance of HHS programs and/or its grantees 
and contractors in carrying out their respective responsibilities, and are intended to 
provide independent assessments of HHS programs and operations in order to reduce 
waste, abuse and mismanagement and to promote economy and efficiency throughout 
the Department 

OFFICE OF INVESTIGATIONS 

The OIG's Office of Investigations (01) conducts criminal, civil, and administrative 
investigations of allegations of wrongdoing in HHS programs or to HHS beneficiaries 
and of unjust enrichment by providers. The investigative efforts of OI lead to criminal 
convictions, administrative sanctions, or civil money penalties. Tht 01 also oversees 
State Medicaid fraud control units which investigate and prosecute fraud and patient 
abuse in the Medicaid program. 

OFFICE OF EVALUATION AND ENSPECnONS 

The OIG's Office of Evaluation and Inspections (OEI) conducts short-term 
management and program evaluations (called inspections) that focus on issues of 
concern to the Department, the Congress, and the public. The findings and 
recommendations contained in the inspections reports gersrate rapid, accurate, and up- 
to-date information on the efficiency, vulnerability, and effectiveness of departmental 
programs. 
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OPPORTUNITIES FOR IMPROVED MANAGEMENT EFFICIENCY 
OF THE HEAD START PROGRAM 

Performance Evaluation and High Risk Determination 



Office of Hu.nan Development Services 



NOTICE 



The designation of financial or management practices as 
questionable or a recommendation for the disallowance of 
costs incurred or claimed, as well as other conclusions 
and recommendations in this report, represent the 
findings and opinions of the OIG Office of Audit 
Ser"vices. Final determinations on these matters will be 
made by authorized HHS operating division officials. 
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The Office of Human Development Services (HDS) 
administers the Head Start program through the 
Administration for Children, Youth and Families (ACYF) . 
The Head Start program was enacted by Congress to provide 
social services to economically disadvantaged preschool 
children and their families. During fiscal year (FY) 
1990 Head Start budgeted $1.2 billion to provide 
comprehensive health, educational, nutritional, social 
and other services to Head Start participants. 

This report is one of three reports that we plan to issue 
relating to opportunities for cost containment and 
improved efficiency of the Head Start program. The other 
two reports deal with HDS' automateci management 
information systems (CIN: A-04-90-C0012) and Head Start 
attendance goals and nonfederal matching requirements 
(CIN: A-04-90-00010) . A draft report on automated 
management information systems was provided to your 
office for comment. 

Performance Evaluation 

Our review showed that HDS has not established or 
implemented procedures necessary to adequately measure 
thfe performance of Head Start agencies. Although 
performance standards have been established (45 CFR 
1304) , HDS has not developed needed criteria to determine 
the extent of an agency's compliance with these 
standards. We believe that HDS should establish and 
implement criteria to rate the iJerformance or Head Start 
agencies. 

A top HDS official indicated that a performance rating 
system was needed. This official also stated that Head 
Start is moving toward a rating system to make management 
decisions and help reduce the workload 

Our review was based on a statistical sample of 2 00 Head 
Start agencies.. We found that HDS had not established 
proct;dures or developed criteria needed to neasure ihe 
quality of an agency's performance. We obtained 
subjective ratings from 6 HDS regional offices for 115 of 
the 200 Head Start agencies selected. The HDS officials 
in Headquarters and 4 regional offices would not rate 
their respective agencies because uniform performance 
rating criteria did not exist. Based on these ratings, 
the operations of 13 agencies were considered to be poor, 
54 were categorized as average, and 48 were determined to 
be good. 
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We believe that a performance rating system would assist 
HDS in its mission to provide quality services to 
preschool children. We also believe that unless a system 
is established, HDS will not be able to properly evaluate 
this complex program. 

Hich Risk Determinations 

The HDS £iwards Head Start grants to agencies to provide 
social services to economically disadvantaged children. 
These agencies must operate their Head Start programs in 
accordance with grant conditions in such areas as program 
performance and financial management and reporting. When 
Head Start agencies do not comply with these conditions, 
they are consic'ered "high risk" by HDS, although very 
rarely does a grantee get classified as high risk as a 
result of program performance. Our review showed that 
HDS needs to formalize its instruction manual procedures 
for carrying out the Head start program. We are 
concerned that the informal process may not provide 
assurances that: (i) methods used to identify high risk 
agencies are uniform throughout the Head Start program; 
(2) conditions that caused a program to be considered 
high risk are corrected in a timely manner; and (3) 
agencies unable to operate successful Head Start programs 
are identified and excluded from participation in the 
program. 

We found that HDS has made efforts to supplement their 
instruction manual by sending guidance, in the form of a 
memorandum, implementing the requirements of 45 CFR part 
92 relating to handling of high risk grantees. We 
believe they are moving in the right direction; however, 
we alsc believe the process would be greatly improved 
through formalization of guidelines relating to such 
areas as time frames for uermination of high risk 
grantees from further participation in Federal programs. 

The Department of Health and Huran Services (HHS) Grants 
Administration Manual and the HDS Grants Administrative 
Staff Manual provide overfill guidance on how granting 
agencies can identify high risk agencies and the actions 
that should be taken once the determination has been 
made, We based our review on requirements prescribed by 
these manuals and HDS high risk lists for the period 
September 1987 through November 1989. We also held 
discussions with HDS regional and Headquarters staff. We 
found that 19 Head Start agencies remained in high rislc 
si:atus for 2 or more ypars, receiving approximately $56 
million in funding during that period. Head Start funds 
ware awarded to some agencies whose management practices 
raise serious questions about their ability to assure 
proper programmatic use and financial stewardship of 
Federal funds. We estimate that approximately $14 



million of the $56 million was underutilized by high risk 
agencies that were later excluded frca participation in 
the Head Start program. 

We recommend that HDS establish and implement improved 
procedures to assure that: 

(1) the extent of an agency's compliance with the 
performance standards is determined and used as 
a basis for establishing uniform ratings for 
agencies; 

(2) agencies whose management practices create high 
risk conditions are uniformly identified; 

(3) high risk conditions, once identified, are 
corrected in a timely manner; and 

(4) high risk agencies that are unable to operate 
succijssful Head Start programs are identified 
and excluded from participation in the program 
in a timely manner. 

In the comments to our draft report dated November 26, 
1990 (Appendix) , HDS agreed with our conclusions 
concerning performance evaluations of Head Start 
agencies. The HDS also agreed that tho current system 
for performance evaluation needed to be strengthened. 
The HDS did not comment on our recommendation to develop 
detailed procedures needed to measure the degree of a 
Head Start agency's compliance with performance standards 
and to use those procedures as a basis for establishing 
uniform ratings for all agencies. 

The comments to the draft report also indicated that HDS 
plans to replace agencies that cannot operate successful 
Head Start programs in a more timely fashion. However, 
HDS did not a^ree with our findings and recommendations 
concerning ,iigh risk determinations. The HDS indicated 
that its' procedures for high risk management are 
included in the HHS Grants Administration Manual and a 
number of policy guidance memoranda. Further, the 
comments indicated that HDS did not believe that changes 
v/ere needed to require the development of definitions of 
high risk agencies or that specific time frames needed to 
be established for resolving high risk conditions. The 
HDS did not comment on our recommendation concerning the 
review of $14 million awarded three high risk head dtart 
agencies. We agree with HDS' proposed plans to replace 
problem Head Start agencies in a more timely fashion, but 
we believe that the other actions recommended in our 
draft report should also be implemented. 
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Tn their written response, HDS provided comments to this 
d?aff report a^^ to Lr draft report on Automated 
Management Information Systems (^-04-90-00012). A 
summary of HDS' comments to a ^f^^^^^f 
^iir T-oiated OIG response is included after the 
cSne?us!ons and rel^SHendations section. f ^^"^ 

?ext of HDS- oonunents applicable to this report is 
included on pages 1 through 3 of the Appendix. 
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INTRODUCTION 



This report provides the results of our rsviJ>w of the 
methodology used by the Office of Uxman Development 
Services (HDS) to measure Head Start performance and 
control high risk Head Start agencies. 

The objectives of our current review wera to determine 
whether the HDS established procedures to: (1) assess 
the performance of Head Start agenciess and (2} uniformly 
identify, monitor and exclude from, participation, if 
necessary, high risk agencies. 

This is one of three reports intended to provide 
information on v/ays HDS can improve the management 
efficiency of the Head Start program. 

Background 

Head Start was established under Title V of the Economic 
Opportunity Act of 1964, as amended. Head Start was 
reauthorized under the Human Services Reauthorization .\ct 
of 1986, which authorized Head Start funds for Fiscal 
Years (FYs) 1987 through 1990. The program was 
reauthorized for 1991. The President's FY 1991 budget 
totaled $1.7 billion for Head Start programs throughout 
the country. The proposed budget included a $500 million 
expansion of the Head Start program to serve an 
additional 180,000 children. 

Head Start is a national program providing comprehensive 
developmental services primarily to low-income preschool 
children, age 3 to the age of compulsory school 
attendance, and their families. To help enrolled 
children achieve their full potential, Head Start 
programs provide comprehensive health, nutritional, 
educational, social and other services. In addition. 
Head Start programs are required to provide for the 
direct participation of parents of enrolled children in 
the development, conduct, and direction of local 
programs. 

Head Start is administered by the Administration for 
Children, Youth and Families (ACYF) within HDS. During 
FYs 1987 through 1989, Head Start annually funded an 
average of 1,895 agencies that provided services to 
approximately 449,000 economically disadvantaged children 
during each of the 3 years. 

Any local public or private nonprofit c.gency with the 
ability to carry out program objectives may be designated 
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by the Secretary as a Head Start grantee agency, A 
grantee agency may then delegate all or a part of its 
responsibility for operating a Head Start program to a 
public or private nonprofit organizationi i.e., a 
delegate agency. 

scope 

The objectives of our review were to determine whether 
the HDS; (1) had established procedures to uniformly 
monitor, assess, and improve the performance of Head 
Start agencies, and (2) exclude from participation in the 
Head Start program, high risk agencies that are not able 
to operate effective Head Start programs. 

Our review was performed at HDS Headquarters and regional 
locations nationwide. The Head Start Performance Evalu- 
ation review was based on a statistical sample of 200 
Head Start agencies selected from the Head Start Cost 
System data base. During the audit, we reviewed Head 
Start performance standards, program profiles, the 
Objective Tracking System (OTS) , best practice and com- 
pliance reports. We also reviewed one regional office's 
internal procedures for "grantee targeting'' (identifica- 
tion of grantees needing assistance) . Wh^^r^ possible, we 
obtained fiscal and programmatic ratings from HDS 
Headquarters and regional office staff. Additionally, we 
held discussions with regional and Headquarters HDS 
staff. 

In evaluating the area of High Risk Determination, we 
examined methods used by HDS to identify high risk 
agencies. We reviewed Title 45 of the Code of Federal 
Regulations, Part 74. We also reviewed the Department of 
Health and Human Services' (HHS) Grants Administration 
Manual (GAM), High Risk Grantees, Chapter 1-05, Monitor- 
ing Discretionary Grants, Chapter 1-13; and HDS Grants 
Administrative Staff Manual, Chapter 8, 

Our review was made in accordance with generally accepted 
Governmental auditing standards. Field work was per- 
formed from October 1988 to December 1989. Our review of 
HDS performance ratings was based on a sample of Head 
Start agencies for FY 1989. Our review of HDS high risk 
determinations was based on those lead Start agencies 
that HDS identified as high risk for the period September 
15, 1987 through November 8, 1989. 



12 



Page 3 - Mary Sheila Gall 
Findings and Recommendations 

Our audit showed that HOS had not established or 
implemented procedures necessary to evaluate the 
performance of Head Start agencies or assure that 
appropriate action is initiated to upgrade or eliminate 
high risk agencies. Although HDS has identified 
functions considered essential for acceptable program 
performance, they had not developed a system to measure 
the degree to which a Head Start agency complies with the 
performance standards. In most instances, HDS top 
managers cannot readily measure Head Start agencies* 
compliance v/ith performance standards. As a result, HDS' 
ability to make informed decisions concerning program 
funding levels, the allocation of discretionary funds or 
changes in program direction is impaired. Because of 
this impairment, HDS cannot adequately 'f.et<<rmine whether 
the $1.2 billion in Head Start funds anA used to provide 
the maximum benefit to the children the program is 
intended to serve. 

The HDS had not specifically identified the number and 
type of indicators that should be present before an 
agency is designated as high risk. The HDS had not 
established specific time limits for agencies to correct 
conditions that led to their high risk do.signations or 
developed criteria that identified high risk agencies 
that were not able to operate effective Head Start 
programs . 

Significant improvement can be made to program oversight 
by: 

o establishing specific procedures to determine the 
extent of Head Start agencies' compliance with 
performance standards and establishing ratings for 
all participating agencies; 

o identifying the number and type of indicators that 
should be present before an agency is designated 
as high risk; 

o establishing specific time limits for agencies to 
correct conditions that led to their high risk 
designations; and 

o developing criteria to identify high risk agencies 
that are not able to operate effective Head Start 
programs , 
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Performance Evaluation 

The HDS has not established a viable performance 
management system prescribing specific criteria needed to 
assure that Head Start agencies provide quality services 
to program participants in a fiscally responsible manner. 
The HDS has not developed a structured evaluation process 
to determine whether Head Start agencies comply with 
performance standards. Additionally, HDS cannot 
determine, with certainty, whether Head Start funds are 
used to provide the maximum benefit to the children the 
program is intended to serve. As a result, HDS' ability 
to make informed decisions concerning program funding 
levels, the allocation of discretionary funds and changes 
in program direction is impaired. We believe that HDS 
should establish and implement criteria to measure the 
quality of services provided by Head Start agencies. 

The HDS does not currently require that Head Start 
agencies be evaluated based on the quality of services 
provided to participants. The HDS developed performance 
standards and indicators, two elements needed to 
establish a performance evaluation system, but only the 
standards have been implemented. Vhe performancr 
standards constitute Head Start policy with which all 
grantee and delegate agencies are required to conform. 
The Head Start standards are accompanied by guidance 
materials that elaborate upon the intent of each 
standard. The guidance is provided for the assistance of 
Head Start programs in interpreting and implementing the 
standards and is not mandatory. Neither the standards 
nor the guidance provides the necessary steps to measure 
the degree to which an agency either complies or does not 
comply with the performance standards. Additionally, 
procedures needed to rank the performance of Head start 
agencies have not been developed. 

Initial development on performance indicators began in 
1978 but was never completed and did not include the 
criteria needed to perform an evaluation of an agency's 
performance. The initial indicators merely listed the 
data that should be considered, identified possible uses 
and provided a plan for implementation. 

The HDS officials stated that uniform performance rating 
criteria was being developed. Currently, the degree to 
which agencies comply with the performance standards is 
determined by regionally developed procedures and 
individual practices. The criteria currently being 
developed consists of 31 Head Start indicators. The HDS 
officials indicated that this criteria would be the basis 
for fiscal and programmatic ratings of Head Start 
agencies. 
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We reviewed the proposed indicators to determine if 
sufficient discriminating criteria was provided to 
measure the qixality of an agency's performance. Our 
review showed that the indicators were classified by 
Head Start performance area (i.e.. Health, Parent 
Involvement/Social Services, Education, and 
Administration/Other) but did not include specifics 
concerning performance measurement. The indicators did 
not identify acceptable ranges or describe individual 
procedures to be followed. These indicators merely 
identified the type of data to be considered and the 
source of the required data. 

The HDS officials indicated that performance measure- 
ment criteria had not been developed. We asked these 
officials to describe the methods HDS used to assess 
the performance of Head Start agencies and to rate our 
iiampled agencies on a fiscal and programmatic basis. 
An HDS regional official indicated that, in addition 
to using prefunding contacts and reviews of grant 
applications, the data maintained on the Program 
Information Report (PIR) system was also used to assure 
that agencies met required performance standards. The 
PIR system contained performance data on the individual 
Head Start agencies and produced two standard reports. 
Compliance and Best Practices, that identified out-of- 
compliance conditions. 

The PIR reports use expected goals and national averages 
to determine compliance with performance standards. The 
reports are used by regional officials to identify 
agencies needing assistance. The PiRs are also the basi 
for decisions concerning the need for increased 
monitoring activity. We were told that each agency 
identified in the PIR reports was notified either by 
memorandum or telephone contact and the areas of non- 
compliance were described. The HDS regional officials 
indicated that, depending on the seriousness of the 
noncompliance issues and the availability of funds, 
additional action may be as little as a phone call or as 
much as an on-site validation assessment. These reports 
assist the regions to identify problem agencies but do 
not measure the quality of services provided by other 
agencies that comply with performance standards. 
Further, there is no evidence to show that these reports 
are consolidated for the use of top HDS managers. 

At the Headquarters level, HDS established, on a FY 
basis, objectives for the Head Start program. The HDS 
regional offices and Headquarters' Indian and Migrant 
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Divisions were required to report, on a quarterly basis, 
their progress in acccmplishing these objectives through 
the HDS Objective Tracking System (OTS) . The objectives 
encompass ACYF's program priorities for the respective 
FY. In FY 1989, the OTS included three goals and 
approximately 28 common objectives. 

One FY 1989 objective wa^ to ensure that Head Start 
agencies complied with all Head Start regulations and 
that agencies provided quality services to Head Start 
enrolles and their families. In order to accomplish this 
objective, regional HDS offices and Headquarters 
divisions were to perform on-site reviews of a 
predetermined number of agencies and report the results 
of these visits through the OTS. While reports generated 
from the OTS were designed to show whether or not 
agancies complied with the performance standards, our 
review showed that most regional office and Headquarter;^ 
division submissions did not address the issue of 
compliance. We found that only 4 of the 12 submissions 
indicated the number of agencies that either complied 
v;ith or were not in compliance with performance 
standards. The omission of this data substantially 
reduces the usefulness of the OTS for measuring an 
agency's compliance with regulations and the quality of 
services provided. 

The HDS officials in Headquarters and four regional 
offices would not rate their respective agencies because 
uniforrfl performance rating criteria did not exist. Five 
of the remainii. six regions provided agency ratings but 
indicated that their ratings were subjective and that HDS 
had not established uniform performance measurement 
criteria. The sixth region's ratings were based on an 
internal regional policy that annually rated its Head 
Start agencies. This system, however, was not used to 
measure agency performance, but rather was used to 
establish uniform procedures for targeting Head Start 
agencies for monitoring as a result of the "debriefing 
process . " 

We obtained HDS' ratings for 115 of the 200 Head Start 
agencies selected for review. These ratings showed that 
the operations of 13 agencies were considered to be poor, 
54 were considered average, and 48 were categorized as 
good Head Start operating agencies. These ratings, 
although based on «nbjective determinations, show that 
significant variawl:^iis exist in the quality of service 
delivery and demonstrate the need for performance 
measurement criteria. If Head Start agencies were rated 
uniformly or consistently, HDS could more effectively 
make decisions concerning grantee participation and 
expected outcome and adjustments. 
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The HDS Headquarters officials indicated that outside 
evaluations of the Head Start program were also used to 
assure that quality services were provided to program 
participants. We asked for a listing of evaluations 
performed during the 4 year period 1987 through 1990. 
To date we have not received this listing or any other 
evidence of outside evaluation efforts. Therefore, we 
cannot determine what effect outside evaluation efforts 
had on assuring that quality services were provided by 
Head Start agencies. 

The FY 1991 budget requests the largest single funding 
increase in the history of the Head Start program - $500 
million or a 36 percent increai?e. This increase in the 
Head Start budget is a key comj^onent in achieving the 
readiness of all children to start school, one of the 
National Education Goals outlined at the President's 
Education Summit. With the expansion of the program, the 
ability to demonstrate or otherwise document that quality 
Head Start services are being provided should be a major 
objective of HDS; we believe the estt.blishment of a 
system to rate the performance of operating agencies is 
essential for the obtainment of that objective. 

The Associate Commissioner of the Head Start Bureau 
realizes that a performance rating system is needed. 
During a presentation made at our pre-audit conference he 
stated that a computerized grant profile would be 
developed and programmatic and fiscal funding judgments 
would be made; once developed. Head Start agencies could 
then be graded. Additionally, he stated that this was 
the management style Head Start was moving toward in 
order to make decisions and necessary program 
improvements. 

We believe that a perfonnance rating system would assist 
HDS in its mission to provide quality servi-^es to 
preschool children. We also believe that it would help 
assure that funds awarded Head Start agencies are 
utilized to provide maximum services to Head Start 
enrolles and their families. 

High Risk Determinations 

Our review at HDS regional and Headquarters offices 
showed that HDS had not established detailed procedures 
needed to effectively manage high risk agencies. The HDS 
had not specifically identified the number and type of 
incicators that should be present before an agency is 
designated as high ris.k. The HDS had not established 
specific time limits for agencies to correct conditions 
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that led to their high risk designations or developed 
criteria that identified high risk agencies that were not 
able to operate effective Head Start programs. Reigional 
HDS officials stated that the HHS Grants Administration 
Manual (GAM) was used to identify high risk agencies but 
that HDS had not established procedures that specifically 
described how high risk determinations were to be made or 
the corrective actions required. A Headquarters HDS 
official indicated that each ragional office was 
responsible for identifying high risk Head Start agencies 
and that the Regional Director of Fiscal and Program 
Operations had the ability and expertise to determine 
whether an agency whould be identified as high risk. The 
current practice of allowing individual regions to make 
these high risk determinations does not assure uniformity 
in identifying and controlling high risk conditions. 

The GAM provides guidance on how granting agencies can 
identify high risk agencies and the actions that should 
be taken once the determination has been made. The GAM 
provides guidance on: (l) indicators of high risk; (2) 
obtaining information for high risk determinations; (3) 
managing risk; and (4) technical assistance to high risk 
agencies. The GAM, however, does not include procedures 
to be used in applying these guidelines. The GAM 
guidelines do not define the number of indicators that 
shoul'i be considered sufficient to classify an agency as 
high risk, nor do these guidelines indicate specific time 
frames allowed for an agency to correct high risk 
conditions. Further, these guidelines do not state or 
indicate when an agency's high risk activity should be 
considered serious enough to render the agency unable to 
operate a Head Start program. The GAM is incorporated 
into Chapter 8 of the HDS Administrative Staff Manual 
which also provides guidance on high risk determination. 
The Administrative Staff Manual defines high risk 
agencies and states HDS' high risk policy, but does not 
identify detailed procedures to be followed for 
identifying and managing high risk agencies. 

In an effort to supplement the GAM and Chapter 8 of 
its staff manual, HDS forwarded to itr Regional 
Administrators a memorandum, dated February 7, 1990, 
that referenced 45 CFR 92. The purpose of 45 CFR 92 is 
to establish uniform administrative requirements for 
grants and cooperative agreements to state, local and 
Indian tribal governments. The issuance of this memo- 
randum demonstrates HDS' recognition that the GAM, as 
well as their staff manual, needs additional criteria for 
Identifying and monitoring high risk non-profit grantees. 
We believe that HDS should movs forward and formalize 
this extended guidance as an initial step to augment 
existing criteria. 
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In a memorandum dat(jd February 3, 1989, a Headquarters 
HDS official recpiired regional offices to provide 
quarterly updates on agencies that remained in a high 
risk status for two or more years. The regional offices 
were directed to provide information on the agencies' 
progress and time frames for corrective action. 
Additionally, in November 1989, HDS performed a limited 
analysis of agencies that remained in high risk status 
for 2 or more years. This analysis showed the current 
status of corrective actions and provided recommendations 
for future activity. Although these actions will assist 
in managing agencies that have remained in high risk 
status for two or more years, they do not provide the 
guidance needed to effectively manage high risk Head 
Start agencies. 

Agencies were designated as high risk for several 
different reasons. These reasons, however, were not 
sufficiently detailed to determine if high risk 
designations were uniformly made across the country. Our 
review did show that some HDS offices were inconsistent 
in their approach and use of high risk determinations. 
Regional HDS offices designated agencies as high risk at 
the point aberrant management practices were identified 
and used this designation to identify agencies that 
needed assistance. On the other hand, Headq-jarters HDS 
officials designated agencies as high risk as a final 
step before the agency was terminated from the Head Start 
program. 

We found that agencies remained in a high risk status for 
periods that ranged from 1 month to 4 and a half years. 
We identified 19 agencies that were considered high risk 
for 2 or more years during the period September 15, 1987 
through November 8, 1989. Approximately $56 million in 
Head Start funds were awarded to these 19 agencies whose 
management practices raised serious questions about their 
ability to assure proper programmatic use and financial 
stewardship of Federal fundsj. In some instances these 
agencies have proven to be so ineffective in their 
operation that HDS ultimately excluded them from 
participation in the Head Start program. 

We found that approximately $19 million was awarded to 
three agencies thit remained in high risk status for more 
than 2 years and were ultimately excluded from 
participation in the Head Start program. We believe that 
approximately $14 million of these funds, awarded during 
the second and sabsequent years the agencies were 
designated as high risk, were not utilized in an 
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efficient manner. Our conclusion th^t these funds were 
under-utilized was based on HOS' requirement that 
instances of non-compliance with performance standards be 
corrected within 1 year. We believe that this 
requirement should also be applicable to high risk 
conditions. The Exhibit to this report shows our 
calculation of the $14 million. 

Conclusions and Recommendations 

Although perfonr.ance standards have been established, 
performance measurement criteria has not been developed 
to assist HDS in determining the extent of an agency's 
compliance with those standards. We believe that a 
performance rating system would augment HDS ' present 
abilities to provide quality services to preschool 
children. 

The HDS procedures for identifying and managing high risk 
agencies do not provide assurance that the: (1) methods 
used to identify high risk agencies are uniform 
throughout the Head Start program; (2) conditions that 
cause cH agency to be considered as high risk are 
corrected in a timely manner; and (3) agencies unable to 
operate successful Head Start programs are identified and 
excluded from participation in the program. 

We recommend that HDS establish and implement procedures 
needed to assure that: 

(1) the extent of Head Start agencies' compliance with 
the performance standards is determined and used 
as a basis for establishing uniform ratings for 
all agencies; 

(2) agencies whose management practices create high 
risk conditions are uniformly identified; 

(3) high risk conditions, once identified, are 
corrected in a timely manner; and 

(4) high risk agencies that are unable to operate 
successful H' id Start programs are identified and 
excluded from participation in the program in a 
timely manner. 

HDS Comments and QIG Response 

Performance Evaluation 

The HDS agreed that an effective system for ev'aluating 
the performance of Head Start agencies was important and 
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also agreed that the current system needed strengthening. 
HDS pointed out that the re-authorization act now 
requires that on-rdte reviews of each grantee he 
performed at leas.t once every three years. Finally, HDS 
indicated that the on-site review process was tt:e most 
important single element in evaluating prograia 
performance and listed six proposed actions to strengthen 
this process. 

OIG ResDor le 

The HDS' comments did not respond to our reccmmendat'' on 
ccncerning developing procedires to determine the degree 
of a Head Start agency's compliance with performance 
standards and establishing uniform ratings for Head Start 
agencies. Unless these type of procedures are 
established, HDS' ability to make informed decisions 
concerning program funding levels, the allocation of 
discretionary funds and changes in program direction is 
impaired. Additionally, without measurement procedures, 
HDS cannot demonstrate or otherwise document that quality 
Head Start services are being provided or assure that 
funds awarded Head Start agencies ar«? used to provide the 
maximum benefit to the children the program is intended 
to serve. Therefore, we believe that HDS roust establish 
and implement detailed procedures to measure the quality 
of services provided participcmts and rate the 
performance of Head Start agencies in addition to 
implementing the corrective actions outlined in the 
written comments. 

High Risk Determination 

In their written response, HDS stated that plans had been 
made to replace agencies that cannot oper=ite successful 
Head Start programs in a more timely fashion. The 
comments indicated that HDS would publish a Notice of 
Proposed Rule Making (NPRM) which would simplify the 
appeals p:rocedures and involve the Departmental Appeals 
Board in hearing appeals. The response also indicated 
that HDS did not believe that suggested changes in its' 
current high risk procedures should bfj made. The HDS 
comments and our responses are summarized below: 

1. The written response £>tated that the draft 
incorrectly reported that HDS did lot have 
detailed procedures for high risk management. The 
response indicated that HDS had procedures in its 
Grants Administration Manual, axpanded through a 
number of policy guidance rc?.moranda . Further, 
HDS ' comments indicated that these procedures were 
in full compliance with HHS policy and that we 
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should revise our draft to clearly state that our 
recommendations suggest that HDS go beyond the 
previously identified procedures. 

OIG Response 

We agree with KDS' proposed plans to help replace problem 
agencies in a more timely fashion, but also believe that 
other measures are necessary to strengthen the management 
of high-risk Head Start agencies. Our review showed that 
HDS had not developed detailed procedures to adequately 
manage high risk agencies. The steps included in the 
Grants Administration Manual (Manual) and requirements of 
HDS • high risk memoranda do not constitute a set of 
detailed procedures for managing high risk conditions. 
The manual clearly states: 

"This chapter provides guidance on how granting 
agencies can identify high risk grantees and the 
actions to be tak^sn once the determination has been 
reached. " 

The manual merely provides examples of some conditions 
that could be used to identify high risk agencies, it 
does not provide the detailed procedures to be used in 
identifying and managing these agencies. 

The available memoranda related to high risk agencies 
required that regional offices provide Headq^aarters HDS 
witn information on the status of high risk agencies that 
remained in a high risk designation for two or more 
years. These memoranda did not provide steps needed to 
identify high risk agencies or to assure that the 
aberrant condition! were corrected in a timely manner. 

We do not believe that HDS' current approach to managing 
high risk agencies is sufficient to assure^: that high risk 
agencies are properly controlled. We believe that "^KDS 
should establish procedures in addition to the guidance 
included in the manual because the data in the manual 
does not contain the detail necessary to assure the 
proper management of high risk agencies. We also believe 
that the need for additional procedures is clearly stated 
in our draft report. 

2 . The HDS stated that instructions and procedures. 

for management of high risk grantees deliberately 
do not require that high risk designations be made 
on the basis of specific numbers or types of 
indicators. The comments further indicate that 
HDS does not believe that the type of quantitative 
precision envisioned in the draft report is 
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applicable to a complex issue such as high risk 
status. Finally, HDS stated that it is preferable 
to continue to rely on the professional judgement 
of HDS Headquarters and regional managers to mrke 
high risk determinations on a case-by-case basis, 
following the guidelines and procedures that are 
contained in existing HDS and Departmental 
instructions . 

OIG Response 

We believe that HDS' policy of not requiring that high 
risk designations be based on specific numbers or types 
of indicators caused the conditions described in our 
report i.e., untimely resolutions of high risk conditions 
and inconsistent treatment of agencies designated as high 
risk between regions and Headquarters. We do not agree 
that it is preferable to continue to rely on individual 
judgments of the regional and Headquarters managers. Our 
review showed that this process resulted in some agencies 
remaining in high risk status for long periods of time 
and has not been effective in correcting aberrant 
conditions or assuring that high risk agencies receive 
consistent treatment. Finally, we do not believe that 
either the guidance found in the HHS Grants 
Administration Manual or the HDS Memoranda provide 
sufficient detail to adequately manage HDS high risk 
agencies. Therefore, we believe that the recommendations 
included in our draft report should be implemented. 

3. The HDS stated that specific time-frames (e.g., 

one year) should not be instituted after which high 
risk grantees would be excluded from participation 
in the program. The HDS indicated that time frames 
were important and had been instituted on a case- 
by-case basis. The HDS did not want an inflexible 
or arbitrary time limit imposed in such a sensitive 
area . 

OIG Response 

Our review showed that time frames were needed to assure 
that aberrant conditions that existed in high risk 
agencies were corrected in a timely manner. We found 
that without such limits, some agencies continued to 
operate unsuccessful programs for extended periods of 
time before they were ultimately excluded from the 
program. We believe that time frames should be 
instituted and that they should be flexible depending on 
circumstances. We also believe that measurable time 
frames should be established. 
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* THIS PERIOD DOES NOT HAVE AN UNDERUTILIZED AMOUNT BECAUSE 
THE GRACE PERIOD OF ONE YEAR TO CORRECT DEFICIENCIES APPLIES. 

# THIS AMOUNT INCLUDES HEAD START FUNDS AWARDED FOR OPERATION 
AND CLOSE OUT OF THE GRANT. 
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TO: 



FROM: 



Richard P. Kusserow 
Inspector General 

Department of Health and Human Services 

Assistant Secretary 

tor Human Development Serviceii 



SUBJECT: 



Response to Draft Reports on Opportunities for Improved 
Management Efficiency of the Head Start Program: Performanci 
-Evaluation and High Risk Determination (A-04-00009) and 
Automated Management Information Systems (A~04-90-00012} 

Thank you for the opportunity to comment on your draft reports 
concerning procedures for the evaluation of Head Start programs and 
the management of "high risk" programs, and Head Start's automated 
information systems. Because the two reports are closely related, we 
are responding to both in this memorandum. 



Performance Evaluation rReoort A«04"00009) 

We agree with the draft report's conclusion about the importance of 
having an effective system for evaluating the performance of local 
Head Start programs. We also agree that the current system for 
performance evaluation must be* strengthened. As Head Star", expands tc 
become a multi-billion dollar program, there must be systems that can 
reliably assess whether grantees are complying with the program's 
Performance Standards. This is especially important because the Head 
Start legislation requires that current grantees receive priority for 
continued funding, if they are performing satisfactorily. 

As requested by the Office of Human Development Services (HDSJ , a new 
provision in the Head Start reauthorization act requires that a full, 
on-site, review be conducted of each grantee at least once every three 
years tc assess whether it meets the program Performance standards. 
The stati-te will require that the results of the review be used, in 
conjunction with other information, to determine whether to continue 
providing non-competitive funding to the grantee or to discontinue 
funding and seek a new agency to serve the community. 
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Although infsrn;aticn ccllecred through automated data systess ?lays ar. 
ir.?crtant role in assessing grantee perf ornjance, the nost iapcrtant 
single element in our systeni for evaluating progranj perfcraance is 
periodic on-site aonitorirjg. We intend to strengthen our monitoring 
system during the coaing year in the following way^: 

o The nuaber of aonitoring reviews will be increased to an 
average of 44 0 per year so that one-third of all prograas 
can be assessed annually. In recent years, resource 
constraints have allowed us only to 2^>onitor as few as 25 
percent of the programs each year. 

o Increased resources will be directed to aonitoring. In TY 
1991, Head Start aonitoring will be a priority use of 
Federal travel funds. A S2 aillion contract, initiated in 
FY 199 0, that supports the use of non-Federal reviewers on 
aonitoring teaas will be continued. Funding for the 
contract will be increased, if necessary. 

o The scope of monitoring visits will be expanded to ensure 
that the reviews are coaprehensive. In addition to 
reviewing compliance wit.h Performance Standards, reviews 
will include fiscal and other administrative and regulatory 
requirements . 

o Procedures will be established to ensure that the reviews 
are consistent across Regional Offices and among grantees. 
Federal staff will be trained to ensure that there are 
common understandings and practices among KDS Regional 
Offices concerning the way in which monitoring instruae.nts 
and other data sources are to be used. Standards will be 
set concerning the numbers of reviewers and the length of 
time required to review programs of various sizes. 

o Monitoring instruments will continue to be revised to 

provide more reliable and consistent findings. The Head 
Start On-site Program Review Instrument (OSPRI) will 
conti.nue to be used during FY 1991. A copy of the OSPRI is 
attached. This instrument contains indicators to help 
measure performance in 51 Xey areas. Further revisions will 
be made to the instrument by FY 1952 to make it easier to 
use a.nd to ensure that the appropriate indicators are 
included. 

o An automated system will be developed to collect and 

sunimarize the results of aonitoring reviews. This will 
enable us to identify specific sub-coaponent areas in which 
prograas across the country are_ performing well or poorly. 
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cSn"l"io"2hat lllTo a±n ""^"itiatad reader to the 

management. That is not cor^^t^ 2n?''S°"""'" f."' '•^?'' '^^^ 
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NOTE: HDS coimnents have been deleted hecarise they pertain to mt-terial that is not 
included in this report. 
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NOTE: HDS comments have been deleted because they pertain to material that is 
included in this report. 
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NOTE: HDS comments have been deleted because they pertain to material that is not 
included in this report. 
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NOTE: HDS comments tiave been deleted because they pertain to material that Is not 
Included In this report. 
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note: HBS cedents bav. b«n deleted because chey pertain to material that is not 
included in this report. 



We look forvard to the opportunity to discuss our plans with you or 
provide additional information. 



Maty Sj(flna Gall 
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